Replication and contradiction of highly cited research papers: a lesson for the Secretary of State for Health?

Trainees seeking membership of the Royal College of Psychiatrists must, through Paper B examinations, demonstrate understanding of critical review, statistical methods and evidence-based practice. This ensures that no psychiatrist progresses through training without an appreciation of how the findings of isolated studies, even those which are highly cited, should be interpreted and applied in clinical practice. The continuing requirement for critical appraisal skills was supported by a recent study of highly cited psychiatric research reporting effective treatments. 1 Tajika et al's 10-year follow-up of 83 papers found that only 16 were replicated, and standardised mean differences had been overestimated by 132%. This is consistent with previous studies reporting a devaluation trend, whereby highly cited clinical research findings are contradicted or weakened over time, 2 and relates to the ongoing problem that positive results are more likely to be published than negative ones. 3 Junior doctors in England currently find themselves in dispute with the government, which has threatened to impose a new contract. 4 The issue is distinct, but not entirely divorceable from a wider debate about 7-day National Health services. The government's argument originates from a single study of mortality rates across the week, 5 although politicians quoted the findings a month before the paper's acceptance by the journal. 6 Commenting on the methodological decision to reclassify Monday and Friday as part of the weekend, in contrast to previous analyses, 7 Professor Alistair Hall queried 'whether such a decision might have been data-driven rather than based on a prior hypothesis-thereby introducing significant bias.' Many are concerned that government rhetoric neglects the authors' own comment that 'it is not possible to ascertain the extent to which these deaths may be preventable; to assume they are avoidable would be rash and misleading.' This prompted the editor of the BMJ to take the unusual step of writing to the Secretary of State for Health, 'to register my concern about the way in which you have publicly misrepresented an academic article published in the BMJ'. 8 The 'weekend effect' has been noted in reverse in psychiatric studies of violent incidents in intensive care 9 and special hospital 10 settings. What interpretation should psychiatrists anticipate that politicians could make of this phenomenon, should it cross their radar? For all of Paper B's laudable focus on evidence-based practice, it sets up today's junior doctor for a disappointing confrontation with political reality. Psychiatrists know better than …